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TitleOflheJnvention 

Automated Caliban For Colored Object Tracking 

Z****^ ^^ef.eMofco.pu—.and.orespecif.cal.y.oa 
This invention relates generally to the Held 

tiv utilized systems track images only in 
. 1( c HoW ever many of the presently utilized sysien 

series of digital frames. However, m y 

<^h svstems are poor at tracking objects tnrougn 
the red-green-blue ("RGB") colorspace. Such systems 

frames in which lighting conditions are changing. 

• P] Pixels are the small discrete elements that 
Dkital images include at least one pixel. Pixels are 

Pach pixel may be classified according to the amount of each 
J to^erconstitutedigitalimages.Bachpixelmy 

' oft hepnmarycolorsofvisiblelight-r e d,greenandblue-(kn 

< tinthepixel If 8 bits of information represent the amount of light fo 

jj that are present in the pixel. R . the 

£ • WsforeacbpixeLthenwUhres^t.otheredcompot-entotanRGBunage, 

S primary colors for each p. ,,,,,,,1) and the darkest red 

S ^htestred.ou.dberepresentedbythenumberZ^Cnb.nary.UUnn) 

, a^rhenun.berOOnHnaryOOOOOOOO). The amounts of green and blue m 
would be represented by the numoer v 

the pixel are also represented in a similar way. 

f P d ereen and blue in an image represented in the RGB 
However, the amounts of red, green ana mu 

„ n different Ughting conditions. For example, in a digital photograph of a 

Kd sweater, m eredco m ponen,of teR OBc P es ^ 

lightmg , »200» in bright lighting conditions, and 40 ,n dun hg 
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,*in g * before, since each of the RGB 

— "^ d -°" ly,he ' S 8 ditions ^r^******"** 

com po»en«sare i n flU encedbyU gttl n g co„dmon S ,..,p 

objectintheROBcotespace. , , , H SV) colorspace. The HSV colorspace, 

h, hue-saturation-value (HSV)coiu v 
Another colorspace.s the hue sa co i orsp ace, 

, „„ rmre sents what humans see. In the to 
ia eonstrast to the ROB colore, better represe 

+rt ;t( . hue the saturation ot its nue, <u 
^hpixeitnaybeciassmedacco^tottshue, ^ 

(v «inapixa H-^s-thewave^on, t,e ^^ ahue 

.presen.dhyc^icaicoordinateshetweenoan ^ ^ 
_ dc , VeUowtsrepresenteahyc—around J 

§ ^ Oreenisrepresente 

| s^onreprcsen. — . ^J^.^^-" 

| on asca,ebetwee„Oa„dl,a — - 3 Averyred p ix e,-haves„much 

« P ,e,wou,d herepresentedoyaSaturat.no,, 

1 .a.tit^tnracappeartohe^ 'J 

1 ^dherepresentedhyaSaturationciosetoO. Huesw 

2 ^^ywithoniyasiightamountofthatHuepresent. ^ 

ona sca,e front 0 to U resents ^ black . Pixels near i are very 

b ri g h,ness.Pixelswi* b ri g h.nessnearOare 

bri8 h,-a.mos, white. ,f the Hue and Saturate are both arou 



the grayscale. 
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. ems^colorspacehasbeenu^fortracBngoperanonsin 

wm puter-vis,nsys,en, T—*— 4te-h *,.-*-.— 

. , ftames Tw „s M da,dtechnk,ueshavebee»usedforcahb ra ..g 

• fft amesofda,a. lnt h=f— .aUof.heOa.anecess^forto 
o.ectmroughasertesotframesof ^ , „ ot env— aUy ^ 

ttaotogishardcodedinafiteorinsourcecode. TtasBc q 

„. correct if the camera, lighting or 
„ hecause the hard-coded data may not he correct 

^ H hried^.eca.ihrattonho.oreach.ame.Whiteth, 
, is assumed that the object to be Backed ^ 

| object is in the right locatron durmg the trme 

f result in bad calibration data. 

§ ^S^^dMS^ ^.oanemhodimentofthepresen, 
3 figure .AiUustratesanHSVcolorspaceaccordmgtoanem 

invention; embodiment of 

,vli„drical H coordinate system accordmg to an emD 
Figure IB illustrates a cylindrical n 

me present invention; ^ fte 

Kgure 2 illustrates a computer setup according to an em 



invention; 
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Figure 3 illustrates a a— embodiment 

capture program is executed by the computer acceding 
program for a video capture prog 

Figure 4B illustrates a close-up vie 

u^; m pnt of the present invention; 
dMaywmdowaceotdingtoanem—of mbodiment „f *e 

present invention; „ o{rox eis in a sampled image, aeeording to an 

Fi gu I e6Amus tt a.esanHSVcolot SP aceofp«els 



X e 
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mbo diment of the present invention; ^ ^ 

♦ .thresholding function according to an emoo 
Figure 6B illustrates a thresholding 

invention; f ^ HSV colorspace 

FiguK6 Ci— a—g-on.penmposedontopof 

to an embodiment of the present invention; 

Figure 6E shows a top view of an HSVcolorsp 

embodiment of the present invention; rf ^ 

F i gU re7Aillus^^ 

P-ntinvention; ^ rfa ^^^ to --^ rf 

Figure 7B illustrates a second part oi 

the present invention; 
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Figure 7C illustrates a tod par, of the ealibration process ac 

according to an embodiment of the 



iccording to an embodiment of 

igure 7C illus*~*~ « th,rcl oan Ui luc Vtt " r " 

the present invention; 

Figure 7D illustrates a first test window's location ; 

present invention; 

A t«t window's 795 location according to an embodiment 
Figure 7E illustrations a second test window 

of the present invention; 

pixel threshold according to an embodiment of the present invention; 

ngure SB mustra.es a probability distribution of pixel data only with respect to 
Saturation according to an embodiment of the present invention; 

1 according to an embodiment of the present invention; 

= invention; _ 
D . • a v iaeo disolay window according to an embodiment ot 

W Figure 10 illustrates an image in a video display wi 

O the present invention; 

of the present invention; 

filteraecordingtoanembodimentofthepresentinvention; 
the present invention; 
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f lto according to an embodiment of , he present invention; 

F ^e 1 4U,us, rat esa h i,o 8 ram„ f *epixe 1 sint h ec„ 1 „m„sa ft er, 1 co lu m„ sh av,n g a 

off e W er, h an3pixe 11 eve 1 s h ave b ee„i g noredacco r d,„ g ,oane mb odime»,o f tHeprese„t 



value 
invention; 



Figure 15 illustrates a histogram 



of the pixels in the rows after all rows having a value of 



invention; 

r uvino a search window according to an 
Figure 16 illustrates an image frame having a searcn w 

embodiment of the present invention; 

P^eniHustratesa^processaccordingtoanemWimcntofthepresent 

invention; and 

en embodiment of the present invention. 

TW lpH Description 

^preferredemnodimentotme — 

p ^^ h .^^-^ te -- ,, ■■- ,,,-ta,,,,,^,,- " 

Figur e 1A i„us tt a,esanHSVco,orspaceaccordin g .oa,em b odimen,o f *e present 
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, Fi8U re,A jt^sm— 

The Saturation coordinate system, S 105. extends 

, , ,00 Thecy.indncalHueeoordinate.HHO.w.psaround.heVcoord 
Value coordinate 100. The cynn aUh eside of 

.. Vaiup level the Saturation level is 1 attnesiu 
, no ,„ a clock-wise radial direction. At any Value level, 

■ H and ^a,«heValuea X is.Therefore,*e.ineardis B ncebe W ee„ 
the "cone" representing Hue, and 0 at the 

, • .iheHcoordiiutesystemincreasesasfteVcoordinateincrease, 

«0» AllpixelsmayberepresentedintheHSVcolorspace. 

360 . AU pixel ob ec , , 0 be 

In th e preferred embodiment, a user utilizes a system to 

llhrates itself to track the object's movements 
, rack ed in a series of digital frames. The system calibrates itself 



CO 

r?\ through the frames. 

sir 



enmouter setup according to an embodiment of the present 
tt Figure 2 illustrates a computer setup ^ 

«,^it^r 700 with a monitor zu:? . i ^ 
f invention. u 1 «hecompu,erse,up,,here,sacompu,er200w 

°- ,,o Thedisplayscreenaiomaybealiouidcrys^ldisplay^ora 

W has a display screen 210. The disp 200 is a digital video camera 

" k rrrm for example. Attached to the computer 200 is rug 

w „ a thode ray tube (CRT), tor exauv ,„inri 
" „ ,, 5may beanydigi«alcamerac„nnec B ble.othecompu.er200. 

H; / ^0^91^ The webcam 21 5 may oe any u & 

(„ebcam)215. Th ^ the computer 200 may digitize an 

mother embodiments, an analog camera could be utilized, 

< „m the analog camera. Additional embodiments may include input 
output video stream from the analog 

s oth er than video cameras, such as streaming video, for example. 

SOU rceso 215isattachedtothe topofthecomputer200. 
In Figure 2, the webcam 215 is attacneo 

, ^S.aybeaphysicalpartofthemonitorormaybephy.cally 

embodiment, the webcam 2 1 5 may be a pny 

separate from the monitor 205. 
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the RGB images ""^^ ^ ^ num ber of algorithms for performing this conversion. 
pro cessingbeg,n, mDUter ,a»guage may perform this function, 

^irhms^eninvi-yanycompnteriang 6 to a calibBt i„„ 

- m e center of the video display ™dov ■ a. 

' ,„ track any movement of a selected object p 

j set, the program to track any 

l WWOW300 ' , ceti ereisno,muchpercep,mledif fe rencehe W eenpi X e,shavi„g 

I mtheHSVcolorspace.Utere, is vet y low. AS- 

« .fferen.H.es.hntsnnilarSatnraUonsandValnes ' ^ 
| ^IwimlowSaturationlooksvery-artoate 

s -r; *i« h -v-.— 

^HcregardlessofmeSamratron. Ther due l0 their 

^ghont.eimagedisp.aywinaowmmennstahlep.x 

c,o S e. U pvie W of*ecalihrationrec,an gl e305wiminmevideo 

^Aillustratesacioseup „ veIltio „. ln Figure 4A, all of 

d , spl aywindow300accordi„g,oanemhodimen,ofhepres 



^ „ very similar Hues, Saturations and Values. 

Ca.ihra.ioumaynotoccunnav.deofta ^ ^ ^ ^ 

^dandthehac^nnd. isshownsub— in, he center of the video 

^rentiocationsonhevrdeodispiaywudo^O ^ lna n 
tec ahhra,ionrecWe30 5 hydra g8 in 8 tUo»ar ^ 

_ emWinrentwherethevideodispiaywtndo^OO ^ 

° • t,na1e 305 may be "4" pixels wide ana P 

| ^im—onrec n e^ ^ ^. Inoth e, 

| emb od ta en.,ho*,hevideodisp 1 ay^300a„ 
a ^enronitoraOSand/orafastprocessorforsantp 

computers 200 havrngareiattvelyi ^ te ^ ^ 

^.eaterdreiastinrehetweenanohjectntov^nfronto 

te ^ diSPl ^" dOW300 ruX r r m .ti-coioredo.ect.OO.ocated^the 
Figu re 4B nhrstra.es a olose-up ^ ^ 
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, • a first colored stripe 405 is located within 

• tanele 305 and within the pixels surrounding it. 
theca lib r ationrectangle305 an k ^^^^ovement through the video 

, thk astern desires to track an object s m 
Whenauserofthissystem During the calibration process, the 

• , W 300 the user must first perform calibration. During 

display window 300, the us -turcuoise" pixels are 

.Preselected In the example shown in Figure 4B, turquoi 
pixels to be tracked are selected. _ ^ 

, ™s if the user wants to track the turquoi f 
loca ted within the calibration rectangle 305. Iftheu 

r* the system to calibrate the pixels for tracking, 
user may prompt the system embod iment of the 

P^S^esap^ec.— p.ocessacco^-e 

fcevideod^aywmdowSOOand ompt 505 ^ syst en, to be gi n c^bration. m 

v have a button which may be pressed to begin 
camera source may have a butto ri ,K brati on A calibration 

„ in a dark room, could be the prompt to begin calibration. 

thepre-object-traclcingp ,„ „,her embodiments, the 

• ^ormessagewhenmeca.ibtationisunsuocessM.^otherem 

gl ven an error messag ^ ^ must again 

^onprocessmayautomatieai.yres.ar,. .fmeproces 
^e^theobieettobe^edintothecaUbraUonrect^e^. 
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• , .00 Thus in order to calibrate the system to track p 

HSV colorspace of the pixels » • ^ ^ " 

, • lc with very low Saturation and the pixels w 
colorspace. Since the pixels with very table pixe ls the unstable pixels 

Q , .waarded To disregard unstable pixels, 

S rf or removed from, the colorspace. The pixels m y 

5§ maybe'nteesholdedoutof.orrem A ^ such as S- V = K, 

. i „f<5 105 and V 100 present in the pixels. * 
| outte sea U pon,he,eve,sofS10 5 a ^ ^ ^ ^ ^ 

f K^apredeterminedcons-.mayoeut.lized 

v „ S 105 and V 100 coordinates are multiple together, and p>x 
W S »V=K,theS105andV gim< jaVof .Swouldbe 

W ■ A For example, ifK - .4, a pixel with an Sof.9anda 

Q less than K are ignored. Forexamp .„, 8 = 72 However, a pixel 

u c "i Q 1 05 and V 100 coordinates is .9 .8 • n 

' — — 

having an S ot .5 anu a 

100 coordinates is .5*.4 = .2. embo diment of the present 

v ,6B illustrates a thresholding function according to an e 

Figure 6B ulustrat function wraps around 

r ^ 600 is formed from the function S*V -K. 

theVaxislOO. The P eH ^ Therefore, a the top of 

c • c "O" at the V axis and 1 at tne n oa 
axis within the HSV cone, S is 0 at the V 
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„ nF, S ure6A .he linear distance between points having the 
^HSVcolorsapc econeshwoninFigure fedependingupontheV , u eeo„ rf .na, 



same Value and Hue, but a different Satnration, vanes 



, m d V is 1- (i-e., » *e flat pa* on the top of the cone), the 
SoifHisaconstantamountandV.s V hmVis »5" 
„ .eVaxisanddteHcootdinatesystenris.axgerthantt.s.henVts.S. 

distance between the V axis ana ,„„ „f the HSV colorspace 

„ 6C illustrates a threshoiding function superimposed on top of the 
F ' 8Ure6C " 1US , tofthepre sentinve„tion.Miofu,epixe 1S be W ee„fte 

^anintageaccordingtoanembodunentofOtepres ^ 

^oOOandaOSformedb^efceshoidingfunettonandUteVax, 

becausetheyareunstable. 

Rgure 6D illustrates an HSV colorsp Th e thresholded HSV colorspace 

* fA, present invention. The mresnonw 
. rem ovedaecordin g .oane m bod im entof.hepresen, urooses only a ^rtion of 

i u „ v 610 and outer boundary 615. For illustrat,ve purposes, only 

I hasaninnerboundary^Oando 

IX the HSV colorspace is shown in Figure 6D. The en 

'« it thp V axis 100 in a radial direction. 

W embodiment ofthe present invention. 

cirface formed by 

1 „esbe W ee n ,ein„ersurfacefor m edby.he,nnerbot„idar y 610and m eou,er 

" theouterboundary615. ,„ ™„cessed by a calibration program. 

Xbeth.shoidedHSVdataformepixelsmaya-enbeprocessedy 

, sln eprogratnde,ern,ines whether the pixels in the object selected by then. 

During processing, the program determined 
betr acKedrhetrac k abi.i.yofase,ec,edob j ec,indiev,deowi„dow300 1 s 

may be tracked. ft w Hue of the pixels selected 

baseduponaprobabilitydistributionoftheS—andoftheHueo 

within the calibration rectangle 305. 
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Inlh e P referredembodime„ ^^^^'"^ , ^ r "^^^' 

pre ae te ™ in ed™ g e>e„ a ,es y s Km wmno,c* rat eon,hese,e«edo b) ec.. 

The standard deviation of the Hue and of the Saturation is also calculated, to ensure that 

»!, UcantrackobjectsofmoderatetohighSaturatton, 
tt ackingplaidorobjec B in«hegrayscale.I.can t racko J 

■n ™lv track an object if the standard deviation of the Hue 
but not grayish objects. The system will only track an j 

and Saturation is below a certain amount. 

. u. vk mav be used. In one embodiment, 
in other embodiments, two calibration rectangles 305 may be 

. a^^^^^ ^^^^ 

.onlvttacks die object, but also knows the angle of the object 
W embodiment, the program not only tracks tneo j 

g relative to static objects in an image, etc. 

S In an additioiAembodiment, two calibration rectangles 305 are simultaneously utilized 

fix ed,c,a,io„sinmevideo\dis,aywi„dow 2 00. h, another embedment, the calibration 



y 

it? 



rectangl 
occur i 
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and tracked. 



A if the Hue and Saturation means and standard 
u a- «,t three frames of data are used. If the Hue an 
embodiment, three name This ensures 

, , f „ rttoee consecutive frames, the program .scalteated. Thrsensu 

The number ot frames per &c 

Una rate of 30 frames/second may be used for 
♦ ♦ thr data to the computer. A sampling rate ot 
camera to get the data to me ^70x240 the 

, d 15frames P ersecon, With a Peripheral Component Interconnect 
- sampling rate may be around 15 frames pe 

S „raer frame and a faster sampling rate are possible. 

S ("PCI") camera, a larger frame anu w^imentof 

J . , Firstth esys t emmustbepromp«ed700,obe g inca 1 ibra t ,o,.n,he 

■ preferred embodiment, a user places an object 

0 • „ „ 100 is being within the calibration rectangle 305, the user 

" calibrationrecWeSO^.h.n.hev.deod.spa, 

A counter W is then loaded 702 with the number 3 , however 

dis cussed in detai. be.ow). The processing desenbed m Bgures 

u, fc, m «, is known as the "series data. 
are acceptable. The data from the three acceptable frames kno 
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^ ■ , a<» already stored in memory. Next, the 

t 70 , the system clears any series pixel data already stor 

m ,o the HSV colorsp embo diment, a 

• The HSV pixel data is then thresholded 708. In in P 
— ^ ' ^s^Vcotnponentsofthepixeida^isutiiizedby^ 
teshoidingtoctiondependentupontheSandV 

, i n other embodiments, the H component may also be used. Du 
thresholding functtor, to other emfo 

, KCBtoHSV theSandYda^isdeterminedmorequtcklythanthe 
conversion from RGB to HSV, th flhenr eferred embodiment, the 

Wore.stnceHisnotusedin.ethresho.dm^ct.onofthepreferre 

v be thresholded while the H coordinates are being calculated. 
HSV colorspace may be thresho 

The system then begins to analyze several testwmdo 

^esthatthesystemusesmebestpossibledataforcMibratioa 

Flgur e 7D i,,Vtesa f ,rs,.e,w,nd„™,.oca,io„accord 1 „g.oa„embo 

-V7 \ * is three pixels wide and two pixels tall (3x2). 

^ / • *; nn Tha claibration rectangle 3Uj is tnree y 

i rfresent invention. inaucuuia 

yresem i ™s the test window is also 3x2. ine 

^ / . -La iv2 calibration rectangle 305, the test w 

b0d ' m en,h T fs , xpMsf ora3x 2 area,„ca,ed,wop, X e.stomele ft a»done 
^emftrstanaly.smeUupors^e.s 

pix e,„pfromtbeloc,iony m eca,ibra«onbox30 5 .As 
ofon e P , x e,be,weenmec\bra.ionbo X 30 5 andme.es,.mdo W7 9 5 .Thesy 
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f • , ThV the system shifts the test window 795 to the next 
pixel data for this group of pixels. Thdn, sys 



location. 



'' \ ■ w, 795 location according to an embodiment 

Fi gure 7\illnstra«io„sasecondtestw 1 ndows7951oca«, 

~5T7 V , r, ire 7Etheen,ire.es.windowasbeenshif.ed 1 rowtofl,en g ht. 

r Af the present inveVtion. In Figure 7E, the am 

W \ , „f9„»elswiththecalibrationrectangle. Afterthe 

^' TO stimethetes.w\dow795hasa„overlapof2p 1 xe,sw 

■ d V,hisgroupofpixe,s,.he,es,window795issh,ftedaga 1 ». Inthe 

:::: 

P \ , ,oS The test window is then shift down one row 

i g \ < • won the right side of the test window 795 

, n tn the left s\) that the two pixels on the rigniM 

the calibration rectangle 305 is ttiapix 

a riaht side of the calibration rWtangle 305. 

oration rectattgle 305 - the upper left-hand corner of the test 

, .,.05 in other embodiments, the different number of 
hand comer of the calibration rectangle 305. In other 

iterations may occur. . $ 
Returning to the process illustrated in Figure 7 A, after the HS 

• the mean Saturation of the pixels in 

708 ,he«es,wind„w709isset,oi.sinitialloca«ion. Then,«hemea„ 

„ ■ ed 710 A.stepTO.ifthemeanSamrauonfallsoutsideofa 
me test window 795 is determined 710. At step 
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♦ in wstem determines that the mean 
• to steo 745 However, if at step 712 the system ae 

Sa ttrati o n doesfa,lw i «h infll eprede.enn I „edra„ge,.hes a ndar 

- 7a^— 

te _-.,— - _t,thenmesys,emde,ermi„es « 

^^.e — de^onofS—^e^ap—e d amount, then *e 

it ^fthpnixels in the test window 795. 
syst em proceeds .0 step 7,8 and determines the meanHue ofthe ptxeis 

The system thenjumps ,0 step 720 as shown in Figure 7B. 

Flgure7Bl „ U stra,esa S e C o„d P a rt o t the— n process a— to an em— 
ofttep rese„,,nventio, lf ,hes y s,e m de,erm,nes7 2 0 to „he mean Hue Mis outside „ f a 

, ,45 However if the Hue data is within the predetemuned range, 
and the system jumps to step 745. However, 

notg o„d,and,hesys,e mJ t m ps t os,e P 7 4 5. lf mes,a„da,ddevia,ionisde t erm 1 ned7 2 4^ 

,ess man me predetermined amount, processing jumps to step 741. 

^mesystemdetermu^ whether arry p.e. data ,s aiready stored in memory or 
.ecurrent— counter*. If there is not, processing jumps, o step 744. 

H.andofS— o f mepi X e 1 ^inmemory,n,isgrea,er,pr„cess 1 „gj U mpstos ttP 74 5 . 
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• a , AtQten744 Dixel data for the test 
K p,aced 744 with the pixel data from the test Wow. A. step 744, p 

window 795 for .he current iteration is stored/replaces in memory. 

„i,»i1>ct the test window 795 is at the last 
Next, at step 745, the system determines whether the test w 

represent— . A , step 74, the system moves the tes, window 795 ,„ the next iocation, 
^umpstostep,.. The system then determines 747 whether any pixe, data has * en 

.-, Hon of counter W If none has, processing jumps to step 704. If pixel 
stored for the current iteration ot counter w 

daB ha S heens,ored,,hec„u»,erWisdecremen,ed74, The system men comhines 749 the 

processed so far. 

N ex„hesys,emde,ermines75 2 whether te s,a„darddevia,„nofHue„fU 1 ep,xeUa. 
■ ^~to~***«-*"'«~~*''~*'' miaL ,fit 

is created 756. 
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• , „„h The graph has an H coordinate 1 10 and 
.hreshCd. Figure 8A illustrates a 3-dunens.ona, graph. ** 

■ „h»hilitv curve 800 and a background pixel threshold 805 
an s coordinate 105. A Gaussran probability curve 

• Mhilitv curve is a symmetrical bell-shaped curve representing 
arebothshown. The Gaussian probability curve 

ft. distribution of Hues and their associated Saturate for the tracked 

belonaine to the background. 

„ in Figure 8A is 3-dimensional. If the H and S axes represent 
The bell curve shape shown in Figure 5A 

a r„„™an curve extends in the Z direction, in a 
XY coordinates in an XYZ coordinate system, the Gaussian curv 

■ ,v .i. h«a Saxes The Gaussian curve, shown in 

direction perpendicular to the plane formed by the H and S axe, 

,, Ml However the bell need not necessarily sweep 
a3 -dimensio„a,s P ace,genera,lyresemb,esabe„. However, 

wtacir „h. Instead.itmightsweepoutanelhpt.Cp,, .notherwords, be «e 
the Gaussian curve in the S direction. 

• + ™ The nartofthe Gaussian curve 
Saturationaccordingtoanembodimentofmepresentinvenuon. Theparto 
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# • 

and an upper*- 820. For a symmetrica, Gaussian curve, the 
dis,ance in terms of Saturation ,eve,s between the mean and each hound is eouivaient. T*e term 
R, (see Figure 9) is hereinafter utilized to refer to the difference in terms of Saturation ieveis 
oetween me Saturation mean 8,0 and each of the iower bound 8,5 and the upper bound 820. 

Figure 8C iUustrates a probabiUty distribution of the pixe, data on,y with respec, to Hue 
according, an embodiment of the present invenUo, The Gaussian curve shown in Figure 8C is 
sim i,ar t o ft eoneshowninFigure 8 B.H„wever,,heGaussiancurveinFigure8Cisnarr„wer 

aiffi ere»cebe M eend,emeanSa,ura,ion8,0andeachof,he,ower Saturation 8,5 bound and, he 
npper Saturation bound 820. The term R 2 (see Figure 9, is hereinafter utihzed ,0 refer ,0 the 
difference in, erms of Hue ,eve,s between the Hue mean 825 andeach of the ,ower bound 830 

and the upper bound 835. 

The probabiUty distribution Hue and Sa,ura,ion of ,he pixe.s representing the seiected 
object in the preferred embodiment has the shape of a Gaussian curve. However, in other 

embodiment, the probabiHty curve may have other shapes. For examp,e, in Figure 8B, the 
d i s.ncebe,wee„,hece„,erSamra,io»8,0andthe.,owerSa,uraUonb„u„d^8,5maybe,ar 6 er 

than the distance between ,he ce„,er Saturation 8,0 and the upper Saturation boundary 820. 
Aiso the background noise thresho.d 805 need not be uniform. ,n fact, the background no.se 
duesho.d 805 ,eve, wi„ ahuost never be uniform u»,ess the background is comply Hack or 
c„mp,e,e,y white. ,n such ahernative embodiment, the noise threshoid may be iagged, for 
example. 
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l ^* ^ -« h ^^*- H - ^< - -, --*' 0,- """ 

inFigU reSA. Two.,,op,areshownisFi g ure, A*p b — I— — ' 
^h. B—"-— — — 

— --^^»-'---*' to - ,fc,ta,4 " ,ta, " h 

Figs SA-SC. ^V***-**"******""™ For the second Hop 905, the 

^ lw ^T*.~~*-«-*-- d,rf ' ,0,, " h - d,,,d 

an upper bound is represented by R4. Each of ,he CHpUca, b.ops shown in the p.xel 

W „ p ,00,a„ y piKe 1 shavi„ g bo t h„ueandSa t ura t io„ 1 eve 1 sfaUin g wi.hinU 1 a t b,opare 
905. 

„ embodiments where the P r„babih,y distributions of Hue and Sa.ura.ion do no, have a 

may not have an elliptical shape. 

To process video frames, ,he system analyzes each pixei, and determines whe.her each 
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t nther words in an embodiment with a 
• fr m<* tn the oixel classification map. In other words, in 
nixel data in a frame to tne pixei ^ 

P . iq 700 nixels (120*160). If any of the pixels fall 

video frame size of 120x160 pixels, there are 19,200 pixels ( 

■ that oixel is represented by a "1" in a bmary map. 

, d ^ereare 19 200 bits of data, one for each pixel. This binary map 

has been completely mapped, there are I*,/ 

may then be utilized for tracking purposes. 

-P- m loo k sfortheob je c 

P ■ • .fnmul the orogram determines that the 

i in^ Tftheob ect is not found, tneprugioi 

around the calibration rectangle 305. It the odj 

I tM difficult .o track a teg e object of uniform odor. 

^^^^^^^^ , , of 

S ^^ound.ThcbackgrounddataisutnWtore.noveareasoftnevtdcow 

the background that resemble the object. 

, ' for the pixels in the frame, applies the data array to 
The system takes the HSV data array for the pixels 
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pixe ,sin,hei,na g e<,4.,6). Each-X-showninFi^.O.epresen.sapixeL Afterthe 
the binary image. 

^ec—d. — — «* fcta * BeCaUSC,he 

. T , Moeat e d a r ound« h eed g eso ft he im a g ea r e. My „ 0 «apa I .ofU,e im a g e,,he Sy ,e m 
preferably filters them out, as explained below. 

represent— . The bisto^ Is created b y su mmi „ g , he "I'sMocated in each of ,e 

^---.^^•»^-' 1 ^' tal,l - ,,,, ' -,,- " ,,iB " , ^ , 

, , 7 Q a low-pass filter is applied to the histogram to reduee the effect of 
of pixels around columns 7-9, a low-pass mra fk 
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.ae^affl-d.suU.o^ayusenayototypeofsui.aHefU.e, 

rows of the binary image. 

^ IJ B I «.^"«---■ rfItab ■ ,l ■* ,fc- " , ''^ ' 

The same low pass filter that was 
^eraccordingtoanemhod.mentofthepresentmven.on.The^ P 

K«1 to Fieure 13 A, resulting in the histogram shown m F lg ure 
applies to Figure 12A was applied to Figure l ja, 
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Fig ure ,4 iuusua.es a histogram of .he pixels in .he o»l- after aU columns having 
inV e„ ti o, A ssho™ fa Fi gUrel2 ,co tam „s 2 A4, 5 ,n, 1 3,an dl 6a 1 ,hado» lyl pi X enn to 

.hose rows have been removed from consideration. 

Hg ure ,5 illustrates a histogram of the pixeis in the rows after aH rows having only 2 or 
fewer pixels have been ignored according to an embodiment of the present invention. As shown 

histogram illus^ed in Figure iS.mepixeisinthoserowshavebeenremovedfrom 

consideration. 

The center of the object is then es.ima,ed from the row and column histograms. The 
S vs.emde.crmi„esthecen,er b a S e d u P on,he r „w„rco 1 u m „havi„g m emost.,"pixe 1 ,Wi.h 

The system therefore determined tha. either column 7 or 8 is the center co.umn for the 
object. In .hepreferre d embodime„.,theee„,eris ra „d„m, y chose„betweenco,u m ns7an d 8 

eachsideof column 7 and 8 is utilised <o determine me center column. Whichever co.um„(i.e., 
c„,umn7or8,has,hemostpixe,s on each side is determined, o be Reenter of *e tt acxed 
o.ec, F orexamp,e,mereare6pixels i „co,umn7and 5 pixe,si„c„lumn,. TWore,the,e 
arel2p ixe,sin.heco,umnsadjacen.tocolumn,The re are3pixe,sineo,um„6and6p 1 xe,s,n 

columns. Therefore, .here are 9 pixels in .he columns adjacent » column 7. Since there are 
^epixelsinmeeolumusadjacenuo column 8, column 8 is determined to be me ccn.er 
column. !f the number ofT pixels located in Une columns adjacentto both columns 7 and 8 had 
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been identical, the system would have considered «he number of "P pixels in each of the 
columns 2 away tan each of columns 7 and 8, and so on until the center was de.erm.ned. 
With respect to the rows of the histogram, row 8 is determined to be the center of the 

image, because it has the most "1" pixels in it. 

Next a "search window" is created for the object in me image frame. Figure 16 

illustrates the video display window 300 having a search window according to an embodiment of 

the object. Binary data for the pixel within the search window is created arrd processed to 
determine the center of the object. The size of the search window is firs, determined. The s, Z e 
ot the search window is dependent upon the number of pixels present in the histogram. The 
estimatedcenteroftheimageisthecenterofdresearchwindow. The center point of the search 
wlndowisthepixelin.he^rowandinthe^column.AsshowninPigureM.therearesix 

- Pixels in the cente, column, column 8. The search window is determined from this data. ,n 
me preferred embodiment, the 3 rows of either side of the center ro, are utilized for the search 
window. Ato.lofsevenrowsarethereforeconsidered. The number of rows in. he search 

^histogram. The search window has an odd number of ^ws. ,f there had been 7 pixels in the 

Therefore, if the histogram has an odd number of pixels in the center columnof the histogram, 
then the total number of rows in the search window are eouivalen. to that number. Conversely, ,f 
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# 

„ , mn8 the center columns, 
were 6 pixels in the column 8, the cerae 

^^e^^ow^oneachs.aeor.ecenterp.e, ^ m 
^epartofthesearchwindow. The large number of ptxelsm 



are 
point 



,, findthecentroidof.heobiect.Thecenttorftsutepomt 

The search window is utthzed to find the cent 

„f all of the T pixels *i<Mn *e search wmdow. In 
^osecoordinatesareflteave^esofaUofthe 

K.r for each "1" pixel is summed. The sum 

. searchwindow^therownumberforeach ordinate of the centroid. Next, 

S , h orTs "inthese a rchwindow.Thisgivestherowcoordmateoft 

S total number of 1 s m uie ^ 
S .erforeach-l-pixelisaddedtogether. The sum » then divided 

ni the column number for each P tP nf the centroid. In the 

- • • This eives the column coordinate of the centro 

| „ umb ero,,., to ,hesearch Wn d„. Th, 
I search^shown.^relMherearc, 

. • *u <;*rn1umn five pixels in the 7 column, v 
W three pixels in the 6 column, n P - VP , in the 11 th column. The 

I pte ,si».heO*co,nr m ,^p i xe,sin,hel0^co tam n,a»donep.xe,.»,e 

S ^dindtecolumndirectionisUterefore Rou „ ded t0 the n eare S , co.u.n 

(((1 . 5)+(3 .6) + (5.7) + (5. 8 ) + (4.9) + (3«>0) + (.«n))^- 7 - 95 - R 
nnmher.ivesacoltrmncoordina.eofS.orthe^comntn. 

Asim i,arproccssisut«tocalc„la,e f hero W coord,„ateof.hecen 

h r pixe,isaddedto g e»e, The sn. .s ,en dWided h y the totalnnmher o f 
number for each 1 P™Hsa , eofthe cen<roid. In Research window 

n,u oives the row coordinate of the cenrroi 
IV in the search wndow. Tins gtves ^ 

„ «i - pixels. There are three pixels m the 5 row, P 
shown in Figure 16, there are 22 1 P«els 
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n^givesarowcoordtaateof^ctheT^w. 
more than one helper object. 

i inn ^undertaken. The HSV data 
invention. First, the aforementioned calibration process 1700 isundertak 

r a 1 70S to the pixel classification map calculated during 
array for the current frame is then applied 1705 to the pixel 
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* a 1 7 1 ft A row histogram is calculated 
lhe cation process. Next, a binary image ,s created 1710. 

estimated 1735. numb er In one embodiment, the 

AcounterQisthenloadedlT^withapredetermmednumbe, 

ter O In other embodiments, other numbers may be loaded mto 
number "5" is loaded into counter Q. in otnere 

" tpdl745 From the search window, the centroid of the 
counter Q. A search window is then created 1745. From 

„ • d 1750 Next the program searches for a helper object 1755. If the 
tracked object is determined 1750. Next, P tfnH l760 

• • i a If the helper object is not found 1760, 
■ r ^ n^n the obiect is tracked 1765. n me ncipc j 
helperobjec,,sfoundl760,theob,e 

counter Q is decremented 1770. If counter Q ,s greater 1775 man zero, 
■i , , 745 „ f ,hesearchwi„dow..fQ i sno,grca,erl77 5 than Z ero,«he 

J be g inn i „ gW i ft mecrea»on 1 745of, h es ^ m5 or 

program determines ,7*0 tha, me object is not w,,h,„ the frame. 
iUsd e,e I minedma,,beob j ec,isno,in,Ke f ,ame,7 8 0,me„data fto m« h ene X , ft ame, 

thereafter acquired 1785. many, 

for the frame being appHed .705 to the pixel classification map. 

enem bodime„,of,eprese„,— . firs, tmage ISOOhas a,rson 1S05 and falhngbox 
1810 . Inlhe „ex„mage,the P erson, 8 05hasrema.„eds— — 

f the imaee but the box 1810 has once agam moved. Again, cross 
is located in the same area of the image, but in 

hairs are superimposed on the center of the box 1810. 
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■ thPQmne of the invention being 

„ ndedciaims rate than .he foregoing description, and a,, changes W h,ch 
indicated by the appended claims, rauie 

• , „f th«. claims are therefore intended to He 
come within the meaning and range of eo.u.va.ency of the clams ar 

embraced therein. 
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